{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "3G1Nx3YLOVaZ"
   },
   "source": [
    "<a target=\"_blank\" href=\"https://colab.research.google.com/github/DeepLabCut/DeepLabCut/blob/main/examples/COLAB/COLAB_DEMO_SuperAnimal.ipynb\">\n",
    "  <img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/>\n",
    "</a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "23v-XAUNQIPY"
   },
   "source": [
    "# DeepLabCut SuperAnimal models\n",
    "\n",
    "![alt text](https://images.squarespace-cdn.com/content/v1/57f6d51c9f74566f55ecf271/1616492373700-PGOAC72IOB6AUE47VTJX/ke17ZwdGBToddI8pDm48kB8JrdUaZR-OSkKLqWQPp_YUqsxRUqqbr1mOJYKfIPR7LoDQ9mXPOjoJoqy81S2I8N_N4V1vUb5AoIIIbLZhVYwL8IeDg6_3B-BRuF4nNrNcQkVuAT7tdErd0wQFEGFSnBqyW03PFN2MN6T6ry5cmXqqA9xITfsbVGDrg_goIDasRCalqV8R3606BuxERAtDaQ/modelzoo.png?format=1000w)\n",
    "\n",
    "http://modelzoo.deeplabcut.org\n",
    "\n",
    "You can use this notebook to analyze videos with pretrained networks from our model zoo - NO local installation of DeepLabCut is needed!\n",
    "\n",
    "- **What you need:** a video of your favorite dog, cat, human, etc: check the list of currently available models here: http://modelzoo.deeplabcut.org\n",
    "\n",
    "- **What to do:** (1) in the top right corner, click \"CONNECT\". Then, just hit run (play icon) on each cell below and follow the instructions!\n",
    "\n",
    "- **Note, if you performance is less that you would like:** firstly check the labeled_video parameters (i.e. \"pcutoff\" that will set the video plotting) - see the end of this notebook.\n",
    "- You can also use the model in your own projects locally. Please be sure to cite the papers for the model, i.e., [Ye et al. 2024](https://arxiv.org/abs/2203.07436) 🎉\n",
    "\n",
    "\n",
    "\n",
    "## **Let's get going: install DeepLabCut into COLAB:**\n",
    "\n",
    "*Also, be sure you are connected to a GPU: go to menu, click Runtime > Change Runtime Type > select \"GPU\"*\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "03ylSyQ4O9Ee"
   },
   "outputs": [],
   "source": [
    "!pip install --pre deeplabcut"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "TguLMTJpQx1_"
   },
   "source": [
    "## PLEASE, click \"restart runtime\" from the output above before proceeding!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "4BejjXKFO2Zg"
   },
   "outputs": [],
   "source": [
    "from pathlib import Path\n",
    "\n",
    "import deeplabcut"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "GXf8N4v28Xqo"
   },
   "source": [
    "## Please select a video you want to run SuperAnimal-X on:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "xXNMNLe6xEBC"
   },
   "outputs": [],
   "source": [
    "from google.colab import files\n",
    "\n",
    "uploaded = files.upload()\n",
    "for filepath, content in uploaded.items():\n",
    "  print(f'User uploaded file \"{filepath}\" with length {len(content)} bytes')\n",
    "\n",
    "video_path = Path(filepath).resolve()\n",
    "\n",
    "# If this cell fails (e.g., when using Safari in place of Google Chrome),\n",
    "# manually upload your video via the Files menu to the left\n",
    "# and define `video_path` yourself with right click > copy path on the video."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "A8sDYMa08f62"
   },
   "source": [
    "## Next select the model you want to use, Quadruped or TopViewMouse\n",
    "- See http://modelzoo.deeplabcut.org/ for more details on these models\n",
    "- The pcutoff is for visualization only, namely only keypoints with a value over what you set are shown. 0 is low confidience, 1 is perfect confidience of the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "ge589yC4v9yX"
   },
   "outputs": [],
   "source": [
    "superanimal_name = \"superanimal_quadruped\" #@param [\"superanimal_topviewmouse\", \"superanimal_quadruped\"]\n",
    "model_name = \"hrnet_w32\" #@param [\"hrnet_w32\", \"resnet_50\"]\n",
    "detector_name = \"fasterrcnn_resnet50_fpn_v2\" #@param [\"fasterrcnn_resnet50_fpn_v2\", \"fasterrcnn_mobilenet_v3_large_fpn\"]\n",
    "pcutoff = 0.15 #@param {type:\"slider\", min:0, max:1, step:0.05}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "zsB0pGtj9Luq"
   },
   "source": [
    "## Okay, let's go! 🐭🦓🐻"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "yqcnEVVSQDC0"
   },
   "outputs": [],
   "source": [
    "videotype = video_path.suffix\n",
    "scale_list = []\n",
    "\n",
    "deeplabcut.video_inference_superanimal(\n",
    "    [video_path],\n",
    "    superanimal_name,\n",
    "    model_name=model_name,\n",
    "    detector_name=detector_name,\n",
    "    videotype=videotype,\n",
    "    video_adapt=True,\n",
    "    scale_list=scale_list,\n",
    "    pcutoff=pcutoff,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "gPLZSBpD34Mj"
   },
   "source": [
    "## Let's view the video in Colab:\n",
    "- otherwise, you can download and look at the video from the left side of your screen! It will end with _labeled.mp4\n",
    "- If your data doesn't work as well as you'd like, consider fine-tuning our model on your data, changing the pcutoff, changing the scale-range\n",
    "(pick values smaller and larger than your video image input size). See our repo for more details."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "ejFJ1Pbg33i6"
   },
   "outputs": [],
   "source": [
    "from base64 import b64encode\n",
    "from IPython.display import HTML\n",
    "import glob\n",
    "\n",
    "# Get the parent directory and stem (filename without extension)\n",
    "directory = video_path.parent\n",
    "basename = video_path.stem\n",
    "\n",
    "# Build the pattern\n",
    "# This uses '*' to allow for any characters between the fixed parts\n",
    "pattern = f\"{basename}*{superanimal_name}*{detector_name}*{model_name}*_labeled_after_adapt.mp4\"\n",
    "\n",
    "# Search for matching files\n",
    "matches = list(directory.glob(pattern))\n",
    "\n",
    "# Choose the first match if it exists\n",
    "labeled_video_path = matches[0] if matches else None\n",
    "\n",
    "view_video = open(labeled_video_path, \"rb\").read()\n",
    "\n",
    "data_url = \"data:video/mp4;base64,\" + b64encode(view_video).decode()\n",
    "HTML(\"\"\"\n",
    "<video width=600 controls>\n",
    "      <source src=\"%s\" type=\"video/mp4\">\n",
    "</video>\n",
    "\"\"\" % data_url)"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "provenance": []
  },
  "gpuClass": "standard",
  "kernelspec": {
   "display_name": "dlc",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13 | packaged by conda-forge | (main, May 27 2022, 17:01:00) \n[Clang 13.0.1 ]"
  },
  "vscode": {
   "interpreter": {
    "hash": "ef00193d8f29a47f592f520086c931b5dd2a83e8a593fa0efe5afff3c413a788"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}
